Task-specific training of reconstruction neural network algorithm for magnetic resonance imaging reconstruction

ABSTRACT

In a computer-implemented method of training a reconstruction neural network algorithm used to reconstruct a Magnetic Resonance Imaging (MRI) image, a prediction of training MRI image is determined based on training MRI raw data and using the reconstruction neural network algorithm. A prediction of a presence or absence of the object in the training MRI image is determined based on the prediction of the training MRI image and using an object-detection algorithm. A loss value is determined based on a first difference between the ground truth of the training MRI image and the prediction of the training MRI image, and further based on a second difference between the ground truth of the presence or absence of the object and the prediction of the presence or absence of the object. Weights of the reconstruction neural network algorithm are adjusted based on the loss value and using a training process.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application claims priority under 35 U.S.C. § 119 toEuropean Patent Application No. EP 21176482.4, filed May 28, 2021, theentire contents of which are incorporated herein by reference.

FIELD

Various examples of the disclosure broadly relate to magnetic resonanceimaging reconstruction of undersampled k-space raw data. Variousexamples of the disclosure more specifically relate to training of arespective reconstruction neural network algorithm.

BACKGROUND

Magnetic resonance imaging (MRI) allows reconstruction of internalanatomic structures of a patient. MRI images depict soft tissue withsignificant contrast. For that purpose, the atomic nuclei are excitedusing radio-frequency (RF) pulses. The excitation can be encoded to aspecific slice using a gradient DC magnetic field. The RF response ofthe relaxing atomic nuclei is measured using RF antennas. Again, DCgradients can be used to obtain spatial resolution. The raw MRI data isavailable in K-space.

Typically, the acquisition of MRI raw data requires significant time;typical acquisition times are in the order of minutes or tens ofminutes. The K-space needs to be sampled with sufficient K-space samplesto allow for reconstruction without ambiguities given a certain field ofview.

To speed up the measurement, the k-space can be undersampled using arespective k-space trajectory. MRI reconstruction can be used toreconstruct the MRI image without aliasing artifacts.

Various techniques for implementing MRI reconstruction are known. Oneprior art technique is referred to as compressed sensing. See, e.g.,Lustig, Michael, David Donoho, and John M. Pauly. “Sparse MRI: Theapplication of compressed sensing for rapid MR imaging.” MagneticResonance in Medicine: An Official Journal of the International Societyfor Magnetic Resonance in Medicine 58.6 (2007): 1182-1195; also seeLustig, Michael, et al. “Compressed sensing MRI.” IEEE signal processingmagazine 25.2 (2008): 72-82.; also Zbontar, Jure, et al. “fastMRl: Anopen dataset and benchmarks for accelerated MRI.” arXiv preprintarXiv:1811.08839 (2018).

Often, such prior art techniques rely on representation of MRI images ina wavelet basis. As described in id., page 13, section “ImageReconstruction”, an optimization problem—typically defined in an

-norm—can be defined. Data consistency can be enforced by adata-consistency operation ensuring that the reconstructed image isdescribed well by the underlying k-space data sparsely sampled. Thedata-consistency operation is also sometimes referred to asdata-fidelity operation or forward-sampling operator. In addition to thedata-consistency operation, oftentimes, a regularization operation isconsidered. The regularization operation is conventionally based on anon-linear

-norm. A classic formulation of the regularization operation is based onsparsity of the MRI image in a transform domain such as a wavelet domainin combination with pseudo-random sampling that can introduce aliasingartifacts that are incoherent in the respective transform domain.Another example would be a Fourier domain, in particular foracquisitions of a dynamically moving target. Another example would betotal variation (TV) used in connection with non-Cartesian k-spacetrajectories such as radial and spiral trajectories.

Based on the data-consistency operation and the regularizationoperation, an iterative optimization can be implemented. The iterativeoptimization can include multiple iterations, each iteration includingthe calculation of the data-consistency operation and the regularizationoperation in an alternating fashion.

Recently, the regularization operation has been implemented via neuralnetwork algorithms (NNs). Here, different iterations of the optimizationare implemented by different layers of the NN. See Hammernik, Kerstin,et al. “Learning a variational network for reconstruction of acceleratedMRI data.” Magnetic resonance in medicine 79.6 (2018): 3055-3071, aswell as Knoll, Florian, et al. “Deep learning methods for parallelmagnetic resonance image reconstruction.” arXiv preprintarXiv:1904.01112 (2019). Such techniques are based on the finding thatwavelet compositions can be expressed as a subset of trainableconvolutions of a deep NN such as a convolutional NN and thatsoft-thresholding can be used as an activation function in the deep NN.

Another example of MRI reconstruction using NNs is described in Xuan,Kai, et al. “Learning MRI k-Space Subsampling Pattern Using ProgressiveWeight Pruning.” International Conference on Medical Image Computing andComputer-Assisted Intervention. Springer, Cham, 2020.

The accuracy of MRI image reconstruction based on a NN depends onaccurate training of the NN.

SUMMARY

The inventors have identified a need for advanced techniques of trainingNNs used for MRI image reconstruction. This, and other needs, may be metby one or more embodiments discussed herein.

Hereinafter, techniques of optimizing the data processing for MRIreconstruction are disclosed. This is achieved through task-specifictraining of a reconstruction NN. More accurate MRI images can bereconstructions. The acquisition time for acquiring the raw MRI data canbe reduced.

A computer-implemented method of training a reconstruction neuralnetwork algorithm is provided. The reconstruction neural networkalgorithm is used to reconstruct am MRI image based on MRI raw data. Thecomputer-implemented method includes obtaining training MRI raw datathat is acquired using a K-space trajectory. The K-space trajectory isundersampling the K-space. The computer-implemented method furtherincludes obtaining a ground truth of a training MRI image that isassociated with the MRI raw data. Additionally, the computer-implementedmethod includes obtaining a ground truth of an object presence orabsence of an object in the training MRI image. The computer-implementedmethod includes, based on the training MRI raw data and using thereconstruction neural network algorithm, determining a prediction of thetraining MRI image. The computer-implemented method further includesdetermining a prediction of a presence or absence of the object in thetraining MRI image based on the prediction of the training MRI image andusing an object-detection algorithm. The computer-implemented methodalso includes determining a loss value based on a first differencebetween the ground truth of the training MRI image and the prediction ofthe training MRI image, as well as further based on a second differencebetween the ground truth of the presence or absence of the object andthe prediction of the presence or absence of the object. Additionally,the computer-implemented method includes adjusting weights of thereconstruction neural network based on the loss value and using atraining process.

A computer-implemented method is provided. The computer-implementedmethod includes obtaining MRI raw data that is acquired using a K-spacetrajectory that is undersampling the K-space. The computer-implementedmethod also includes determining a prediction of an MRI image using areconstruction neural network algorithm. The computer-implemented methodfurther includes determining a prediction of a presence or absence of anobject in the MRI image using an object-detection algorithm. Thereconstruction neural network algorithm has been trained using thecomputer-implemented method of training described above.

A computer program or a computer-program or a computer-readable storagemedium including program code is provided. The program code can beloaded and executed by at least one processor. Upon loading andexecuting the program code, the at least one processor performs acomputer-implemented method of training a reconstruction neural networkalgorithm. The reconstruction neural network algorithm is used toreconstruct am MRI image based on MRI raw data. The computer-implementedmethod includes obtaining training MRI raw data that is acquired using aK-space trajectory. The K-space trajectory is undersampling the K-space.The computer-implemented method further includes obtaining a groundtruth of a training MRI image that is associated with the MRI raw data.Additionally, the computer-implemented method includes obtaining aground truth of an object presence or absence of an object in thetraining MRI image. The computer-implemented method includes, based onthe training MRI raw data and using the reconstruction neural networkalgorithm, determining a prediction of the training MRI image. Thecomputer-implemented method further includes determining a prediction ofa presence or absence of the object in the training MRI image based onthe prediction of the training MRI image and using an object-detectionalgorithm. The computer-implemented method also includes determining aloss value based on a first difference between the ground truth of thetraining MRI image and the prediction of the training MRI image, as wellas further based on a second difference between the ground truth of thepresence or absence of the object and the prediction of the presence orabsence of the object. Additionally, the computer-implemented methodincludes adjusting weights of the reconstruction neural network based onthe loss value and using a training process.

A device for training a reconstruction neural network algorithm isprovided. The reconstruction neural network algorithm is used toreconstruct an MRI image based on MRI raw data. The device includes aprocessor that is configured to obtain training MRI raw data. Thetraining MRI raw data is acquired using a K-space trajectory that isundersampling the K-space. The processor is further configured to obtaina ground truth of a training MRI image that is associated with the MRIraw data. The processor is further configured to obtain a ground truthof an object presence or absence of an object in the training MRI image.The processor is further configured to determine a prediction of thetraining MRI image based on the training MRI raw data and using thereconstruction neural network algorithm. The processor is furtherconfigured to determine a prediction of a presence or absence of theobject in the training MRI image based on the prediction of the trainingMRI image and using an object-detection algorithm. The processor isfurther configured to determine a loss value based on a first differencebetween the ground truth of the training MRI image and the prediction ofthe training MRI image, as well as further based on a second differencebetween the ground truth of the presence or absence of the object in theprediction of the presence or absence of the object. The processor isfurther configured to adjust weights of the reconstruction neuralnetwork algorithm based on the loss value and using a training process.

A further device is provided. The device includes a processor that isconfigured to obtain MRI raw data acquired using a k-space trajectorythat is undersampling the k-space. The processor is further configuredto determine a prediction of an MRI image using a reconstruction neuralnetwork algorithm, and to determine a prediction of a presence orabsence of an object in the MIR image using an object-detectionalgorithm. Herein, the reconstruction neural network algorithm has beentrained using the provided computer-implemented method of training areconstruction neural network algorithm.

It is to be understood that the features mentioned above and those yetto be explained below may be used not only in the respectivecombinations indicated, but also in other combinations or in isolationwithout departing from the scope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be discussed in more detailwith regard to the drawings in which:

FIG. 1 schematically illustrates an MRI device according to variousexamples.

FIG. 2 schematically illustrates a device according to various examples.

FIG. 3 is a flowchart of a method according to various examples.

FIG. 4 illustrates data processing according to various examples.

DETAILED DESCRIPTION

Some examples of the present disclosure generally provide for aplurality of circuits or other electrical devices. All references to thecircuits and other electrical devices and the functionality provided byeach are not intended to be limited to encompassing only what isillustrated and described herein. While particular labels may beassigned to the various circuits or other electrical devices disclosed,such labels are not intended to limit the scope of operation for thecircuits and the other electrical devices. Such circuits and otherelectrical devices may be combined with each other and/or separated inany manner based on the particular type of electrical implementationthat is desired. It is recognized that any circuit or other electricaldevice disclosed herein may include any number of microcontrollers, agraphics processor unit (GPU), integrated circuits, memory devices(e.g., FLASH, random access memory (RAM), read only memory (ROM),electrically programmable read only memory (EPROM), electricallyerasable programmable read only memory (EEPROM), or other suitablevariants thereof), and software which co-act with one another to performoperation(s) disclosed herein. In addition, any one or more of theelectrical devices may be configured to execute a program code that isembodied in a non-transitory computer readable medium programmed toperform any number of the functions as disclosed.

In the following, embodiments of the present invention will be describedin detail with reference to the accompanying drawings. It is to beunderstood that the following description of embodiments is not to betaken in a limiting sense. The scope of the present invention is notintended to be limited by the embodiments described hereinafter or bythe drawings, which are taken to be illustrative only.

The drawings are to be regarded as being schematic representations andelements illustrated in the drawings are not necessarily shown to scale.Rather, the various elements are represented such that their functionand general purpose become apparent to a person skilled in the art. Anyconnection or coupling between functional blocks, devices, components,or other physical or functional units shown in the drawings or describedherein may also be implemented by an indirect connection or coupling. Acoupling between components may also be established over a wirelessconnection. Functional blocks may be implemented in hardware, firmware,software, or a combination thereof.

Various techniques described herein generally relate to MRI imaging. MRIraw data is acquired in k-space by sampling k-space. Parallel imagingcan be applied. Here, MRI data is acquired using an array of receivercoils having a predefined spatial sensitivity. The MRI raw data issparsely sampled in k-space, i.e., MRI data is acquired below theNyquist threshold for a given field of view. This is sometimes referredto as undersampling k-space. According to various examples, the MRImeasurement datasets may be obtained using an undersampling trajectory.When acquiring MRI measurement datasets using an undersamplingtrajectory for certain k-space locations, samples are not obtained andthe missing information is reconstructed later on. A so-calledacceleration factor R is indicative of the fraction of those k-spacelocations along the undersampling trajectory for which no raw datasamples are acquired. Larger (smaller) acceleration factors may resultin a shorter (longer) scan times.

Then, MRI reconstruction is employed to reconstruct an MRI image(reconstructed MRI image) without or having reduced aliasing artifacts.The MRI reconstruction often relies on predetermined or calibrated coilsensitivity maps (CSMs) of multiple receiver coils of the RF receiver ofthe MRI device are used.

Various techniques disclosed herein rely on MRI reconstruction usingNNs. Oftentimes, a trained NN can outperform conventionalreconstructions (including iterative approaches such as CompressedSensing) when applied to a known/trained acquisition. This also goes bythe name of deep learning (DL) reconstruction.

As a general rule, the NN employed in the various examples can be aconvolution NN. Here, each convolutional layer has a certain kernelwhich is convoluted against the input values. A specific example that isoften employed is the U-net, see Ronneberger, Olaf, Philipp Fischer, andThomas Brox. “U-net: Convolutional networks for biomedical imagesegmentation.” International Conference on Medical image computing andcomputer-assisted intervention. Springer, Cham, 2015. The U-net employsskip-connections between hidden layers and down-sampling and up-samplingof feature maps.

As a general rule, various options are available of implementing MRIreconstruction and, more specifically, MRI reconstruction using the NN.Some of these options are summarized in TAB. 1 below.

TABLE 1 Various options for using one or more reconstruction NNs forreconstructing MRI images from undersampled k-space data. These optionscan be combined with each other. Brief description Example details ISingle In one example, it would be possible to feed-forward use the NNto remove aliasing artefacts aliasing from a spatial-domainrepresentation of removal the raw MRI data. Here, a single forward- passof the NN may be used to remove the aliasing artefacts. Thespatial-domain representation of the raw MRI data can be obtainedthrough a Fourier transform of the raw MRI data. See, e.g., Xuan, Kai,et al. “Learning MRI k-Space Subsampling Pattern Using ProgressiveWeight Pruning.” International Conference on Medical Image Computing andComputer-Assisted Intervention. Springer, Cham, 2020. II Fourier In afurther example, it would be possible transform to implement the Fouriertransform of the raw MRI data using a NN. A Fourier transformation is alinear transform, and as such may be expressed by a special form of afully connected layer of the NN. The weights are the initialized suchthat the fully connected layer performs a Fourier transformation of theinput. By making the weights trainable, the Fourier transformation canthen be adapted to the task at hand. A general example of suchreconstruction is disclosed in: Zhu, Bo, et al. “Image reconstruction bydomain-transform manifold learning.” Nature 555.7697 (2018): 487-492.III Iterative In yet a further example, an iterative optimizationoptimization can include (i) a for aliasing regularization operator -that is removal implemented by a trained NN such as a Convolutional NN(CNN) - for filtering of the input MRI dataset using convolutions andnon-linear activations; and (ii) a data-consistency operator (sometimesreferred to as forward-sampling operator or data-fidelity operator) forcomputation of an MRI forward model to assure agreement of thereconstructed MRI image with the MRI raw data. This approach of using aniterative optimization together with a deep-NN having layers associatedwith each iteration goes by the name of a variational NN (VNN). Thecomplete network is also called an unrolled image reconstructionnetwork, or simply unrolled network. Multiple iterations of (i) and (ii)iteratively refine the reconstructed MRI image, wherein an appropriateoptimization technique, for example a gradient descent optimizationtechnique or Landweber iterations, or prima-dual method, or alternatingdirection method of multipliers as known in the art, may be used tooptimize parameters from iteration to iteration, i.e., to minimize agoal function including the regularization operator and thedata-consistency operator. Such optimization technique may define partsof the data-consistency operation. The data-consistency operation can bebased on the squared  

 ₂-norm of the difference between measured data and synthesized datausing a signal model. A gradient can be considered, in accordance withthe optimization technique. In particular for decorrelated data withGaussian noise this can be a good choice. The signal model can beSENSE-type and, in particular, may rely on predefined CSMs. The CSMs canbe calculated separately. Examples of such techniques are disclosed inSee Hammernik, Kerstin, et al. “Learning a variational network forreconstruction of accelerated MRI data.” Magnetic resonance in medicine79.6 (2018): 3055-3071, as well as Knoll, Florian, et al. “Deep learningmethods for parallel magnetic resonance image reconstruction.” arXivpreprint arXiv: 1904.01112 (2019).

As will be appreciated, various options are available for leveraging MRIreconstruction using an NN. By using the NN in the context of the MRIreconstruction, an increased image quality of the respectivereconstructed MRI dataset may be provided. A reduced noise amplificationand reduced image artifacts can be obtained, in comparison with theconventional MRI reconstruction techniques. The natural image appearancemay be better preserved using NN, e.g., without causing significantblurring in comparison to techniques with hand-crafted regularizationoperators. Conventional compressed sensing techniques may be slow andmay result in less natural looking images. Using the NN, faster imagereconstruction may be achieved using a predefined number of iterationsof the NN. The reconstruction time is usually several orders ofmagnitude faster than in other iterative methods.

According to various examples, it is possible to increase the accuracyand quality of the MRI reconstruction using NNs.

Hereinafter, various techniques will be described that facilitatetask-specific MRI reconstruction. Typically, the MRI imaging is appliedin the context of a specific task, e.g., a certain diagnostic purpose.For instance, MRI imaging may be applied to detect a stroke region inthe brain. MRI imaging may be applied to detect a teninopathy. MRIimaging may be applied to detect tumors or torn ligaments. Spinal corddefects can be detected. Depending on the specific task, differentregions or body parts are imaged. Depending on the specific task,different objects are to be detected.

Various examples are based on the finding that task-specific MRIreconstruction can be facilitated by considering, at the training of thereconstruction NN-cf. TAB. 1-, a task-specific loss value. According tovarious examples, a task-specific loss value is considered whenadjusting weights of the reconstruction NN.

In particular, it is possible to determine a contribution to the lossvalue that depends on whether or not a certain object—typically definedin a task-specific manner—is present or absent in the prediction of theMRI image obtained from the reconstruction of the raw MRI data.According to various examples, it is possible that the loss value isdetermined based on a difference between a ground truth of the presenceor absence of an object and a prediction of the presence or absence ofthe object as obtained from an object-detection algorithm.

Thereby, it is possible to increase the accuracy and quality of the MRIreconstruction using NNs.

FIG. 1 depicts aspects with respect to an MRI device 100. The MRI device100 includes a magnet 110, which defines a bore 111. The magnet 110 mayprovide a DC magnetic field of, e.g., one to eight Tesla along itslongitudinal axis. The DC magnetic field may align the magnetization ofthe patient 101 along the longitudinal axis. The patient 101 may bemoved into the bore via a movable table 102.

The MRI device 100 also includes a gradient system 140 for creatingspatially-varying magnetic gradient fields (gradients) used forspatially encoding MRI data. Typically, the gradient system 140 includesat least three gradient coils 141 that are arranged orthogonal to eachother and may be controlled individually. By applying gradient pulses tothe gradient coils 141, it is possible to apply gradients along certaindirections. The gradients may be used for slice selection(slice-selection gradients), frequency encoding (readout gradients), andphase encoding along one or more phase-encoding directions(phase-encoding gradients). Hereinafter, the slice-selection directionwill be defined as being aligned along the Z-axis; the readout directionwill be defined as being aligned with the X-axis; and a firstphase-encoding direction as being aligned with the Y-axis. A secondphase-encoding direction may be aligned with the Z-axis. The directionsalong which the various gradients are applied are not necessarily inparallel with the axes defined by the coils 141. Rather, it is possiblethat these directions are defined by a certain k-space trajectory,which, in turn, may be defined by certain requirements of the respectiveMRI sequence and/or based on anatomic properties of the patient 101.

As a general rule, the k-space trajectory may also be adjusted during atraining of the MRI reconstructions, to match the k-space trajectorywith the reconstruction data processing. Such techniques will beexplained in connection with FIG. 3 : box 3030.

For preparation and/or excitation of the magnetization polarized/alignedwith the DC magnetic field, RF pulses may be applied. For this, an RFcoil assembly 121 is provided which is capable of applying an RF pulsesuch as an inversion pulse or an excitation pulse or a refocusing pulse.While the inversion pulse generally inverts the direction of thelongitudinal magnetization, excitation pulses may create transversalmagnetization.

For creating such RF pulses, an RF transmitter 131 is connected via a RFswitch 130 with the coil assembly 121. Via a RF receiver 132, it ispossible to detect signals of the magnetization relaxing back into therelaxation position aligned with the DC magnetic field. In particular,it is possible to detect echoes; echoes may be formed by applying one ormore RF pulses (spin echo) and/or by applying one or more gradients(gradient echo). The magnetization may inductively coupled with the coilassembly 121 for this purpose. Thereby, raw MRI data in k-space isacquired; according to various examples, the associated MRI measurementdatasets including the MRI data may be post-processed in order to obtainimages. Such post-processing may include a Fourier Transform fromk-space to image space. Such post-processing may also include MRIreconstruction configured to avoid motion artifacts.

Generally, it would be possible to use separate coil assemblies forapplying RF pulses on the one hand side and for acquiring MRI data onthe other hand side (not shown in FIG. 1 ). For example, for applying RFpulses a comparably large body coil 121 may be used; while for acquiringMRI data a surface coil assembly including an array of comparably smallcoils could be used. For example, the surface coil assembly couldinclude 32 individual RF coils arranged as receiver coil array 139 andthereby facilitate spatially-offset coil sensitivities. Respective CSMscan be defined.

The MRI device 100 further includes a human machine interface 150, e.g.,a screen, a keyboard, a mouse, etc. Via the human machine interface 150,a user input may be detected and output to the user may be implemented.For example, via the human machine interface 150, it is possible to setcertain configuration parameters for the MRI sequences to be applied.

The MRI device 100 further includes a processing unit (simply processor)161. The processor 161 may include a GPU and/or a CPU. The processor 161may implement various control functionality with respect to theoperation of the MRI device 100, e.g., based on program code loaded froma memory 162. For example, the processor 161 could implement a sequencecontrol for time-synchronized operation of the gradient system 140, theRF transmitter 131, and the RF receiver 132. The processor 161 may alsobe configured to implement an MRI reconstruction, i.e., implementpost-processing for MRI reconstruction of MRI images based on MRImeasurement datasets.

It is not required in all scenarios that processor 161 implementspost-processing for reconstruction of the MRI images. In other examples,it would be possible that respective functionalities implemented by aseparate device, such as the one as illustrated in FIG. 2 .

FIG. 2 schematically illustrates a device 90 according to variousexamples. The device 90 includes a processing unit/processor 91 and amemory 92. The processor 91 can obtain MRI Raw data via an interface 93,e.g., from a hospital database, a computer-readable storage medium, ordirectly from an MRI device 100 as discussed in connection with FIG. 1 .Upon loading program code from the memory 92, the processor 91 canpost-process the MRI raw data, to reconstruct an MRI image.Specifically, the processor 91 can employ a reconstruction NN-cf. NN—todetermine a prediction of the MRI image. Also, the processor 91 cantrain the reconstruction NN.

Details with respect to such training of the reconstruction NN areillustrated in connection with FIG. 3 .

FIG. 3 is a flowchart of a method according to various examples. A NN istrained to accurately reconstruct MRI images based on MRI raw data thathas been acquired using K-space trajectories that are undersampling theK-space. The NN can be employed in the context according to one of theexamples of TAB. 1. The method of FIG. 1 could be executed by onon-device processor such as the processor 161 of the MRI device 100 inthe example of FIG. 1 . It would also be possible that the method ofFIG. 1 is executed by a separate device such as the device 90. Forinstance, the method of FIG. 3 could be executed by the processor 92 ofthe device 90 upon loading program code from the memory 93.

The method commences at box 3005. At box 3005, training MRI raw data isobtained. The training MRI raw data may be obtained from an MRI devicesuch as the MRI device 100. The training MRI data may be obtained from adatabase, e.g., a picture archiving system of a hospital or a centraldatabase on the Internet.

Obtaining the data may generally include loading the respective data viaan interface or from a memory, or even controlling another device toacquire the data.

The method proceeds at box 3010; here, a ground truth of a training MRIimage that is associated with the training MRI data is obtained. Thetraining MRI image can thus resemble the artifact-free image-domainrepresentation of the MRI raw data.

As a general rule, various options are available for obtaining thetraining MRI image. For instance, it would be possible to fully samplethe K-space, i.e., using a K-space trajectory that is not undersamplingthe K-space. Based on such full sampling of the K-space, it would bepossible to determine the training MRI image. Then, it would be possibleto discard certain K-space samples such that the undersampling of theK-space according to the K-space trajectory is obtained for therespective training MRI data. In such an approach it is also possible tosynthesize different raw MRI data for different K-space trajectories.This can be helpful for scenarios in which the K-space trajectory itselfis adjusted during the training, as will be later-on explained inconnection with box 3030.

Next, a ground truth of object presence or object absence is obtained atbox 3015. For instance, a respective label indicated for of whether acertain anatomical structure or condition is visible in the training MRIimage may be obtained. It would be possible to use supervised learning.I.e., respective labels may be annotated by an expert, e.g., aclinician.

The object can be defined in a task-specific manner. I.e., depending onthe specific radiology task, a different object may be defined.

As a general rule, object presence or object absence may be determinedfor multiple different object classes. I.e., for certain image bodyregion, there may be multiple candidate objects that could bepotentially visible in the training MRI image. For each of thesecandidate objects, would be possible to determine object presence orobject absence as ground truth.

Example objects include: tendon rupture; vertebral fracture; strokeregion; thrombosis; etc..

At box 3015, it would also be possible to obtain a ground truth foradditional information pertaining to the object. For instance, it wouldbe possible to obtain a ground truth of an object location of the objectin the training MRI image.

As a general rule, various options are available for specifying theobject location. For instance, a center of the object could bespecified. It would be possible to use bounding boxes, e.g., rectangularor square structures or even 3D structures of minimum size that enclosethe entire object.

At box 3020, a prediction of the training MRI image is determined usingthe NN. Here, the NN in its current training states, i.e., using thecurrently set weights, is used.

Then, also at box 3020, a prediction of the presence or absence of theobject in the training MRI image is determined, using anobject-detection algorithm that operates based on the prediction of thetraining MRI image also determined at box 3020.

As a general rule, various options are available for implementing theobject-detection algorithm. Classical feature-based detection algorithmscould be used. It would also be possible to use an object-detection NN.Here, various kinds and types of object-detection NNs can be used asgenerally known to the skilled person. An example would be described in:Lin, Tsung-Yi, et al. “Focal loss for dense object detection.”Proceedings of the IEEE international conference on computer vision.2017. The object-detection NN is used in its current training state,i.e., using the currently set weights.

Optionally, where applicable, it would also be possible to determine aprediction of the location of the object in the training MRI image basedon the training MRI image and using the object-detection algorithm.

Then, the method commences at box 3025. Here, a loss value isdetermined. The loss value is sometimes also referred to as cost value.

To do so, the predictions of box 3020 are compared with the respectiveground truth. This would include, e.g., determining the loss value basedon a first difference between the ground truth of the training MRI imageand the prediction of the training MRI image, and further based on asecond difference between the ground truth of the presence or absence ofthe object in the prediction of the presence or absence of the object.

For instance, the first difference could be a pixel wise differencesquared and then a sum across all pixels could be formed.

Optionally, a third difference can be taken into account whendetermining the loss value, the third difference being between theground truth of the object location of the object and the prediction ofthe object location of the object.

As will be appreciated, various task-specific contributions to the lossvalue have been disclosed, i.e., presence or absence of the object andobject location. As a general rule, further or different task-specificcontributions to the loss value would be conceivable, e.g., a confidencelevel of the object-detection algorithm.

More generally, the loss value could be expressed as: L(x,θ)=Detection(R(x, theta))+Reconstruction(R(x, theta))+Undersampling(x,theta).

Here x refers to the raw MRI data in k-space, θ refers to the learnableparameters (i.e., weights of the respective networks as well asparameters for sampling pattern determination), and R(x, theta) is thereconstructed MRI image.

“Detection” is based on the difference between the prediction of theabsence or presence of the object and optionally the prediction of theobject location, as well as the respective ground truth. More generally,this contribution can be task-specific.

“Reconstruction” is the image-based contribution, i.e., the differencebetween the prediction of the MRI image and the ground truth of the MRIimage. For instance, a comparison between corresponding pixels could beimplemented.

“Undersampling” is a contribution to the loss value that can beoptionally considered. It is associated with the k-space trajectory. Forinstance, this contribution could penalize longer acquisition times.I.e., it would be possible that the loss value is further based on anacquisition time associated with the K-space trajectory. Thereby, duringthe training, specifically an iterative training process, the amount ofK-space samples can be reduced. The time required to acquire the imagecan be reduced.

The undersampling contribution is generally optional. In some examples,it would be possible that the undersampling contribution is onlyconsidered in connection with adjusting the trajectory used to samplethe K-space, while the detection loss and the reconstruction loss areused to adjust the parameters of the NN. in such a scenario, it is notrequired to combine the undersampling contribution with the detectionand reconstruction contribution into a single loss value. Rather, twoseparate loss values can be used, one adjusting the K-space trajectory,and one for adjusting the weights of the NN.

It is then possible to adjust weights at least of the NN, at box 3030.This is based on the loss value in using a training process. Varioustraining processes generally known to the skilled person in the art canbe used, e.g., back-propagation: Here, the gradient of the loss functioncan be calculated with respect to the weights of the network. Thisenables an iterative optimization technique—see FIG. 3 , where boxes3020, 3025, and 3030 can be iteratively executed—employing a gradientdescent optimization. This can be used where a set of training MRI rawdata and respective ground truths is obtained.

For a scenario in which a NN is used to implement the object-detectionalgorithm, it would be possible to not only adjust the weights of theNN, but also of the object-detection NN, based on the loss value, at box3030. Specifically, it would be possible to employ end-to-end thetraining of the NN and the object-detection NN. Here, a gradient of theloss function can be determined for the object detection NN can befurther propagated into the reconstruction NN. More generally, theweights of the reconstruction NN and the object-detection NN arecoherently adjusted, i.e., in a combined fashion and coupled to eachother. This increases the accuracy of the training.

Above, examples have been illustrated in which parameters of thereconstruction NN and optionally the object detection NN are adjusted atbox 3030. It is optionally possible that at box 3030, hyperparameters ofthe reconstruction NN and/or even of the MRI measurement are adjusted.

For instance, it would be possible to implement network slimming. Here,it would be possible that some weights of the reconstruction NN arefixedly set to 0. Specifically, it would be possible to force suchweights of the reconstruction NN to zero that fall below a certainpredefined threshold. Respective techniques are generally described inXuan, Kai, et al. “Learning MRI k-Space Subsampling Pattern UsingProgressive Weight Pruning.” International Conference on Medical ImageComputing and Computer-Assisted Intervention. Springer, Cham, 2020. Forillustration, such techniques may be specifically employed where thereconstruction NN is used to implement a Fourier transform, cf. TAB. 1:example II. By network slimming, the complexity of the calculation canbe reduced, reducing computational resources.

In a further example, it would be possible to slim-alternatively oradditionally to slimming the reconstruction NN—the K-space trajectory.Specifically, it could be checked which weights of the input layer ofthe reconstruction NN have values that are smaller than a certainpredefined threshold. Then, corresponding K-space positions may beremoved from the K-space trajectory. Thereby, the acquisition timerequired to sample the K-space can be reduced.

As a general rule, alternatively or additionally, and specificallybeyond slimming of the K-space trajectory, it would be generallypossible that the training process adjusts the K-space trajectory duringthe training process and based on the loss value.

I.e., it would be possible to re-arrange sampling points in K-space. Thesampling point density could be adjusted. The orientation and/or shapeof the K-space trajectory could be adjusted. For instance, differentshapes of K-space trajectories may be chosen, e.g., radial, spiral,cartesian, random. Data points may be added or removed.

There are various options available for implementing said adjusting ofthe K-space trajectory using the training process. For instance, wherean iterative optimization is used to implement the training process,random adjustments can be made from iteration to iteration in therespective impact on the loss value can be observed. It would also bepossible to make directed adjustments, e.g., based on the weights of theinput layer of the reconstruction NN.

By such adjusting of the K-space trajectory, it can be ensured that suchspatial frequencies are sampled that have a high impact on the accuratereconstruction of the MRI image. At the same time, unnecessary orlow-impact spatial frequencies may not be sampled, or respective regionsmay be sampled less densely.

As a general rule, when adjusting the K-space trajectory, it would bepossible to observe one or more predefined constraints. Such constraintscan be based on hardware limitations, e.g., with respect to switchingspeeds, or strength of DC magnetic field gradients. Thereby, aphysically feasible K-space trajectory can be obtained.

Once the training has been completed, the reconstruction NN can be usedto make predictions during inference, i.e., where no ground truth isavailable.

FIG. 4 illustrates the data processing for MRI reconstruction. Areconstruction NN 111 is employed that can be trained using the methodof FIG. 3 .

The MRI raw data, e.g., training MRI raw data 131, corresponds to datasamples that have been acquired using a K-space trajectory 151undersampling the K-space 150.

While in FIG. 4 a Cartesian K-space trajectory as illustrated, as ageneral rule, various other types and forms of K-space trajectories 151may be used, e.g., radial sampling, circular sampling, or randomsampling. The K-space trajectory 151 can also be adjusted using thetraining process.

The (training) MRI raw data 131 is then passed to a reconstruction NN111. For instance, the reconstruction NN 111—e.g., a CNN—can include afirst sub-reconstruction NN implementing a Fourier transform of thetraining MRI raw data 131 to the image domain, and a secondsub-reconstruction NN implementing removal of aliasing artefacts.

Thereby, a reconstruction of a (training) MRI image 132 in image domain160 is obtained. The (training) MRI image 132 depicts an object 139,here a stroke region.

Based on the prediction of the (training) MRI image 131, theobject-detection algorithm 121 can determine the prediction of presenceor absence 134 and optionally of location 133 of the object 139. Thelocation 133 is illustrated with the bounding box and the presence orabsence 134 with the respective label.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, components, regions,layers, and/or sections, these elements, components, regions, layers,and/or sections, should not be limited by these terms. These terms areonly used to distinguish one element from another. For example, a firstelement could be termed a second element, and, similarly, a secondelement could be termed a first element, without departing from thescope of example embodiments. As used herein, the term “and/or,”includes any and all combinations of one or more of the associatedlisted items. The phrase “at least one of” has the same meaning as“and/or”.

Spatially relative terms, such as “beneath,” “below,” “lower,” “under,”“above,” “upper,” and the like, may be used herein for ease ofdescription to describe one element or feature's relationship to anotherelement(s) or feature(s) as illustrated in the figures. It will beunderstood that the spatially relative terms are intended to encompassdifferent orientations of the device in use or operation in addition tothe orientation depicted in the figures. For example, if the device inthe figures is turned over, elements described as “below,” “beneath,” or“under,” other elements or features would then be oriented “above” theother elements or features. Thus, the example terms “below” and “under”may encompass both an orientation of above and below. The device may beotherwise oriented (rotated 90 degrees or at other orientations) and thespatially relative descriptors used herein interpreted accordingly. Inaddition, when an element is referred to as being “between” twoelements, the element may be the only element between the two elements,or one or more other intervening elements may be present.

Spatial and functional relationships between elements (for example,between modules) are described using various terms, including “on,”“connected,” “engaged,” “interfaced,” and “coupled.” Unless explicitlydescribed as being “direct,” when a relationship between first andsecond elements is described in the disclosure, that relationshipencompasses a direct relationship where no other intervening elementsare present between the first and second elements, and also an indirectrelationship where one or more intervening elements are present (eitherspatially or functionally) between the first and second elements. Incontrast, when an element is referred to as being “directly” on,connected, engaged, interfaced, or coupled to another element, there areno intervening elements present. Other words used to describe therelationship between elements should be interpreted in a like fashion(e.g., “between,” versus “directly between,” “adjacent,” versus“directly adjacent,” etc.).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of exampleembodiments. As used herein, the singular forms “a,” “an,” and “the,”are intended to include the plural forms as well, unless the contextclearly indicates otherwise. As used herein, the terms “and/or” and “atleast one of” include any and all combinations of one or more of theassociated listed items. It will be further understood that the terms“comprises,” “comprising,” “includes,” and/or “including,” when usedherein, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof. As used herein,the term “and/or” includes any and all combinations of one or more ofthe associated listed items. Expressions such as “at least one of,” whenpreceding a list of elements, modify the entire list of elements and donot modify the individual elements of the list. Also, the term “example”is intended to refer to an example or illustration.

It should also be noted that in some alternative implementations, thefunctions/acts noted may occur out of the order noted in the figures.For example, two figures shown in succession may in fact be executedsubstantially concurrently or may sometimes be executed in the reverseorder, depending upon the functionality/acts involved.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which example embodiments belong. Itwill be further understood that terms, e.g., those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

It is noted that some example embodiments may be described withreference to acts and symbolic representations of operations (e.g., inthe form of flow charts, flow diagrams, data flow diagrams, structurediagrams, block diagrams, etc.) that may be implemented in conjunctionwith units and/or devices discussed above. Although discussed in aparticularly manner, a function or operation specified in a specificblock may be performed differently from the flow specified in aflowchart, flow diagram, etc. For example, functions or operationsillustrated as being performed serially in two consecutive blocks mayactually be performed simultaneously, or in some cases be performed inreverse order. Although the flowcharts describe the operations assequential processes, many of the operations may be performed inparallel, concurrently or simultaneously. In addition, the order ofoperations may be re-arranged. The processes may be terminated whentheir operations are completed, but may also have additional steps notincluded in the figure. The processes may correspond to methods,functions, procedures, subroutines, subprograms, etc.

Specific structural and functional details disclosed herein are merelyrepresentative for purposes of describing example embodiments. Thepresent invention may, however, be embodied in many alternate forms andshould not be construed as limited to only the embodiments set forthherein.

In addition, or alternative, to that discussed above, units and/ordevices according to one or more example embodiments may be implementedusing hardware, software, and/or a combination thereof. For example,hardware devices may be implemented using processing circuitry such as,but not limited to, a processor, Central Processing Unit (CPU), acontroller, an arithmetic logic unit (ALU), a digital signal processor,a microcomputer, a field programmable gate array (FPGA), aSystem-on-Chip (SoC), a programmable logic unit, a microprocessor, orany other device capable of responding to and executing instructions ina defined manner. Portions of the example embodiments and correspondingdetailed description may be presented in terms of software, oralgorithms and symbolic representations of operation on data bits withina computer memory. These descriptions and representations are the onesby which those of ordinary skill in the art effectively convey thesubstance of their work to others of ordinary skill in the art. Analgorithm, as the term is used here, and as it is used generally, isconceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of optical, electrical, or magnetic signals capable of beingstored, transferred, combined, compared, and otherwise manipulated. Ithas proven convenient at times, principally for reasons of common usage,to refer to these signals as bits, values, elements, symbols,characters, terms, numbers, or the like.

It should be borne in mind that all of these and similar terms are to beassociated with the appropriate physical quantities and are merelyconvenient labels applied to these quantities. Unless specificallystated otherwise, or as is apparent from the discussion, terms such as“processing” or “computing” or “calculating” or “determining” of“displaying” or the like, refer to the action and processes of acomputer system, or similar electronic computing device/hardware, thatmanipulates and transforms data represented as physical, electronicquantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

In this application, including the definitions below, the term ‘module’or the term ‘controller’ may be replaced with the term ‘circuit.’ Theterm ‘module’ may refer to, be part of, or include processor hardware(shared, dedicated, or group) that executes code and memory hardware(shared, dedicated, or group) that stores code executed by the processorhardware.

The module may include one or more interface circuits. In some examples,the interface circuits may include wired or wireless interfaces that areconnected to a local area network (LAN), the Internet, a wide areanetwork (WAN), or combinations thereof. The functionality of any givenmodule of the present disclosure may be distributed among multiplemodules that are connected via interface circuits. For example, multiplemodules may allow load balancing. In a further example, a server (alsoknown as remote, or cloud) module may accomplish some functionality onbehalf of a client module.

Software may include a computer program, program code, instructions, orsome combination thereof, for independently or collectively instructingor configuring a hardware device to operate as desired. The computerprogram and/or program code may include program or computer-readableinstructions, software components, software modules, data files, datastructures, and/or the like, capable of being implemented by one or morehardware devices, such as one or more of the hardware devices mentionedabove. Examples of program code include both machine code produced by acompiler and higher level program code that is executed using aninterpreter.

For example, when a hardware device is a computer processing device(e.g., a processor, Central Processing Unit (CPU), a controller, anarithmetic logic unit (ALU), a digital signal processor, amicrocomputer, a microprocessor, etc.), the computer processing devicemay be configured to carry out program code by performing arithmetical,logical, and input/output operations, according to the program code.Once the program code is loaded into a computer processing device, thecomputer processing device may be programmed to perform the programcode, thereby transforming the computer processing device into a specialpurpose computer processing device. In a more specific example, when theprogram code is loaded into a processor, the processor becomesprogrammed to perform the program code and operations correspondingthereto, thereby transforming the processor into a special purposeprocessor.

Software and/or data may be embodied permanently or temporarily in anytype of machine, component, physical or virtual equipment, or computerstorage medium or device, capable of providing instructions or data to,or being interpreted by, a hardware device. The software also may bedistributed over network coupled computer systems so that the softwareis stored and executed in a distributed fashion. In particular, forexample, software and data may be stored by one or more computerreadable recording mediums, including the tangible or non-transitorycomputer-readable storage media discussed herein.

Even further, any of the disclosed methods may be embodied in the formof a program or software. The program or software may be stored on anon-transitory computer readable medium and is adapted to perform anyone of the aforementioned methods when run on a computer device (adevice including a processor). Thus, the non-transitory, tangiblecomputer readable medium, is adapted to store information and is adaptedto interact with a data processing facility or computer device toexecute the program of any of the above mentioned embodiments and/or toperform the method of any of the above mentioned embodiments.

Example embodiments may be described with reference to acts and symbolicrepresentations of operations (e.g., in the form of flow charts, flowdiagrams, data flow diagrams, structure diagrams, block diagrams, etc.)that may be implemented in conjunction with units and/or devicesdiscussed in more detail below. Although discussed in a particularlymanner, a function or operation specified in a specific block may beperformed differently from the flow specified in a flowchart, flowdiagram, etc. For example, functions or operations illustrated as beingperformed serially in two consecutive blocks may actually be performedsimultaneously, or in some cases be performed in reverse order.

According to one or more example embodiments, computer processingdevices may be described as including various functional units thatperform various operations and/or functions to increase the clarity ofthe description. However, computer processing devices are not intendedto be limited to these functional units. For example, in one or moreexample embodiments, the various operations and/or functions of thefunctional units may be performed by other ones of the functional units.Further, the computer processing devices may perform the operationsand/or functions of the various functional units without sub-dividingthe operations and/or functions of the computer processing units intothese various functional units.

Units and/or devices according to one or more example embodiments mayalso include one or more storage devices. The one or more storagedevices may be tangible or non-transitory computer-readable storagemedia, such as random access memory (RAM), read only memory (ROM), apermanent mass storage device (such as a disk drive), solid state (e.g.,NAND flash) device, and/or any other like data storage mechanism capableof storing and recording data. The one or more storage devices may beconfigured to store computer programs, program code, instructions, orsome combination thereof, for one or more operating systems and/or forimplementing the example embodiments described herein. The computerprograms, program code, instructions, or some combination thereof, mayalso be loaded from a separate computer readable storage medium into theone or more storage devices and/or one or more computer processingdevices using a drive mechanism. Such separate computer readable storagemedium may include a Universal Serial Bus (USB) flash drive, a memorystick, a Blu-ray/DVD/CD-ROM drive, a memory card, and/or other likecomputer readable storage media. The computer programs, program code,instructions, or some combination thereof, may be loaded into the one ormore storage devices and/or the one or more computer processing devicesfrom a remote data storage device via a network interface, rather thanvia a local computer readable storage medium. Additionally, the computerprograms, program code, instructions, or some combination thereof, maybe loaded into the one or more storage devices and/or the one or moreprocessors from a remote computing system that is configured to transferand/or distribute the computer programs, program code, instructions, orsome combination thereof, over a network. The remote computing systemmay transfer and/or distribute the computer programs, program code,instructions, or some combination thereof, via a wired interface, an airinterface, and/or any other like medium.

The one or more hardware devices, the one or more storage devices,and/or the computer programs, program code, instructions, or somecombination thereof, may be specially designed and constructed for thepurposes of the example embodiments, or they may be known devices thatare altered and/or modified for the purposes of example embodiments.

A hardware device, such as a computer processing device, may run anoperating system (OS) and one or more software applications that run onthe OS. The computer processing device also may access, store,manipulate, process, and create data in response to execution of thesoftware. For simplicity, one or more example embodiments may beexemplified as a computer processing device or processor; however, oneskilled in the art will appreciate that a hardware device may includemultiple processing elements or processors and multiple types ofprocessing elements or processors. For example, a hardware device mayinclude multiple processors or a processor and a controller. Inaddition, other processing configurations are possible, such as parallelprocessors.

The computer programs include processor-executable instructions that arestored on at least one non-transitory computer-readable medium (memory).The computer programs may also include or rely on stored data. Thecomputer programs may encompass a basic input/output system (BIOS) thatinteracts with hardware of the special purpose computer, device driversthat interact with particular devices of the special purpose computer,one or more operating systems, user applications, background services,background applications, etc. As such, the one or more processors may beconfigured to execute the processor executable instructions.

The computer programs may include: (i) descriptive text to be parsed,such as HTML (hypertext markup language) or XML (extensible markuplanguage), (ii) assembly code, (iii) object code generated from sourcecode by a compiler, (iv) source code for execution by an interpreter,(v) source code for compilation and execution by a just-in-timecompiler, etc. As examples only, source code may be written using syntaxfrom languages including C, C++, C#, Objective-C, Haskell, Go, SQL, R,Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5,Ada, ASP (active server pages), PHP, Scala, Eiffel, Smalltalk, Erlang,Ruby, Flash®, Visual Basic®, Lua, and Python®.

Further, at least one example embodiment relates to the non-transitorycomputer-readable storage medium including electronically readablecontrol information (processor executable instructions) stored thereon,configured in such that when the storage medium is used in a controllerof a device, at least one embodiment of the method may be carried out.

The computer readable medium or storage medium may be a built-in mediuminstalled inside a computer device main body or a removable mediumarranged so that it can be separated from the computer device main body.The term computer-readable medium, as used herein, does not encompasstransitory electrical or electromagnetic signals propagating through amedium (such as on a carrier wave); the term computer-readable medium istherefore considered tangible and non-transitory. Non-limiting examplesof the non-transitory computer-readable medium include, but are notlimited to, rewriteable non-volatile memory devices (including, forexample flash memory devices, erasable programmable read-only memorydevices, or a mask read-only memory devices); volatile memory devices(including, for example static random access memory devices or a dynamicrandom access memory devices); magnetic storage media (including, forexample an analog or digital magnetic tape or a hard disk drive); andoptical storage media (including, for example a CD, a DVD, or a Blu-rayDisc). Examples of the media with a built-in rewriteable non-volatilememory, include but are not limited to memory cards; and media with abuilt-in ROM, including but not limited to ROM cassettes; etc.Furthermore, various information regarding stored images, for example,property information, may be stored in any other form, or it may beprovided in other ways.

The term code, as used above, may include software, firmware, and/ormicrocode, and may refer to programs, routines, functions, classes, datastructures, and/or objects. Shared processor hardware encompasses asingle microprocessor that executes some or all code from multiplemodules. Group processor hardware encompasses a microprocessor that, incombination with additional microprocessors, executes some or all codefrom one or more modules. References to multiple microprocessorsencompass multiple microprocessors on discrete dies, multiplemicroprocessors on a single die, multiple cores of a singlemicroprocessor, multiple threads of a single microprocessor, or acombination of the above.

Shared memory hardware encompasses a single memory device that storessome or all code from multiple modules. Group memory hardwareencompasses a memory device that, in combination with other memorydevices, stores some or all code from one or more modules.

The term memory hardware is a subset of the term computer-readablemedium. The term computer-readable medium, as used herein, does notencompass transitory electrical or electromagnetic signals propagatingthrough a medium (such as on a carrier wave); the term computer-readablemedium is therefore considered tangible and non-transitory. Non-limitingexamples of the non-transitory computer-readable medium include, but arenot limited to, rewriteable non-volatile memory devices (including, forexample flash memory devices, erasable programmable read-only memorydevices, or a mask read-only memory devices); volatile memory devices(including, for example static random access memory devices or a dynamicrandom access memory devices); magnetic storage media (including, forexample an analog or digital magnetic tape or a hard disk drive); andoptical storage media (including, for example a CD, a DVD, or a Blu-rayDisc). Examples of the media with a built-in rewriteable non-volatilememory, include but are not limited to memory cards; and media with abuilt-in ROM, including but not limited to ROM cassettes; etc.Furthermore, various information regarding stored images, for example,property information, may be stored in any other form, or it may beprovided in other ways.

The apparatuses and methods described in this application may bepartially or fully implemented by a special purpose computer created byconfiguring a general purpose computer to execute one or more particularfunctions embodied in computer programs. The functional blocks andflowchart elements described above serve as software specifications,which can be translated into the computer programs by the routine workof a skilled technician or programmer.

Although described with reference to specific examples and drawings,modifications, additions and substitutions of example embodiments may bevariously made according to the description by those of ordinary skillin the art. For example, the described techniques may be performed in anorder different with that of the methods described, and/or componentssuch as the described system, architecture, devices, circuit, and thelike, may be connected or combined to be different from theabove-described methods, or results may be appropriately achieved byother components or equivalents.

Although the present invention has been shown and described with respectto certain example embodiments, equivalents and modifications will occurto others skilled in the art upon the reading and understanding of thespecification. The present invention includes all such equivalents andmodifications and is limited only by the scope of the appended claims.

What is claimed is:
 1. A computer-implemented method of training areconstruction neural network algorithm used to reconstruct a MagneticResonance Imaging (MRI) image based on MRI raw data, thecomputer-implemented method comprising: obtaining training MRI raw dataacquired using a k-space trajectory that is undersampling the k-space,obtaining a ground truth of a training MRI image associated with thetraining MRI raw data, obtaining a ground truth of a presence or absenceof an object in the training MRI image, determining a prediction of thetraining MRI image based on the training MRI raw data and using thereconstruction neural network algorithm, determining a prediction of thepresence or absence of the object in the training MRI image based on theprediction of the training MRI image and using an object-detectionalgorithm, determining a loss value based on (i) a first differencebetween the ground truth of the training MRI image and the prediction ofthe training MRI image and (ii) a second difference between the groundtruth of the presence or absence of the object and the prediction of thepresence or absence of the object, and adjusting weights of thereconstruction neural network algorithm based on the loss value andusing a training process.
 2. The computer-implemented method of claim 1,further comprising: obtaining a ground truth of a location of the objectin the training MRI image, and determining a prediction of the locationof the object in the training MRI image based on the training MRI imageand using the object-detection algorithm, wherein the loss value isfurther determined based on a third difference between the ground truthof the location of the object and the prediction of the location of theobject.
 3. The computer-implemented method of claim 1, wherein theobject-detection algorithm is an object-detection neural networkalgorithm, and the method further includes adjusting weights of theobject-detection neural network algorithm based on the loss value andusing the training process.
 4. The computer-implemented method of claim3, wherein the training process is an end-to-end training processcoherently adjusting the weights of the reconstruction neural networkalgorithm and the weights of the object-detection neural networkalgorithm.
 5. The computer-implemented method of claim 1, furthercomprising: slimming at least one of the reconstruction neural networkalgorithm or the k-space trajectory using the training process.
 6. Thecomputer-implemented method of claim 1, further comprising: adjustingthe k-space trajectory based on the loss value and using the trainingprocess.
 7. The computer-implemented method of claim 1, wherein the lossvalue is further based on an acquisition time associated with thek-space trajectory.
 8. A computer-implemented method, comprising:obtaining MRI raw data acquired using a k-space trajectory that isundersampling the k-space, determining a prediction of an MRI imageusing a reconstruction neural network algorithm, and determining, usingan object-detection algorithm, a prediction of a presence or absence ofan object in the MRI image, wherein the reconstruction neural networkalgorithm has been trained using the computer-implemented method ofclaim
 1. 9. A device for training a reconstruction neural networkalgorithm used to reconstruct a Magnetic Resonance Imaging (MRI) imagebased on MRI raw data, the device comprising: one or more processorsconfigured to obtain training MRI raw data acquired using a k-spacetrajectory that is undersampling the k-space, obtain a ground truth of atraining MRI image associated with the training MRI raw data, obtain aground truth of a presence or absence of an object in the training MRIimage, determine a prediction of the training MRI image based on thetraining MRI raw data and using the reconstruction neural networkalgorithm, determine a prediction of the presence or absence of theobject in the training MRI image based on the prediction of the trainingMRI image and using an object-detection algorithm, determine a lossvalue based on (i) a first difference between the ground truth of thetraining MRI image and the prediction of the training MRI image and (ii)a second difference between the ground truth of the presence or absenceof the object and the prediction of the presence or absence of theobject, and adjust weights of the reconstruction neural networkalgorithm based on the loss value and using a training process.
 10. Adevice comprising one or more processors configured to perform thecomputer-implemented method of claim
 1. 11. A device comprising: one ormore processors configured to obtain MRI raw data acquired using ak-space trajectory that is undersampling the k-space, determine aprediction of an MRI image using a reconstruction neural networkalgorithm, and determine a prediction of a presence or absence of anobject in the MRI image using an object-detection algorithm, wherein thereconstruction neural network algorithm has been trained using thecomputer-implemented method of claim
 1. 12. A non-transitory computerprogram product comprising instructions which, when executed by acomputer, cause the computer to carry out the method of claim
 1. 13. Anon-transitory computer-readable storage medium comprising instructionswhich, when executed by a computer, cause the computer to carry out themethod of claim
 1. 14. The computer-implemented method of claim 2,wherein the object-detection algorithm is an object-detection neuralnetwork algorithm, and the method further includes adjusting weights ofthe object-detection neural network algorithm based on the loss valueand using the training process.
 15. The computer-implemented method ofclaim 2, further comprising: slimming at least one of the reconstructionneural network algorithm or the k-space trajectory using the trainingprocess.
 16. The computer-implemented method of claim 3, furthercomprising: slimming at least one of the reconstruction neural networkalgorithm or the k-space trajectory using the training process.
 17. Thecomputer-implemented method of claim 4, further comprising: slimming atleast one of the reconstruction neural network algorithm or the k-spacetrajectory using the training process.
 18. The computer-implementedmethod of claim 2, further comprising: adjusting the k-space trajectorybased on the loss value and using the training process.
 19. Thecomputer-implemented method of claim 3, further comprising: adjustingthe k-space trajectory based on the loss value and using the trainingprocess.
 20. The computer-implemented method of claim 4, furthercomprising: adjusting the k-space trajectory based on the loss value andusing the training process.